The present invention relates in general to novel configurations of trainable resistive crosspoint devices, which are referred to herein as resistive processing units (RPUs). More specifically, the present invention relates to artificial neural networks (ANNs) formed from crossbar arrays of two-terminal RPUs that provide local data storage and local data processing without the need for additional processing elements beyond the two-terminal RPU, thereby accelerating the ANN's ability to learn and implement algorithms such as online neural network training, matrix inversion, matrix decomposition and the like.
Technical problems such as character recognition and image recognition by a computer are known to be well handled by machine-learning techniques. “Machine learning” is used to broadly describe a primary function of electronic systems that learn from data. In machine learning and cognitive science, neural networks are a family of statistical learning models inspired by the biological neural networks of animals, and in particular the brain. Neural networks can be used to estimate or approximate systems and functions that depend on a large number of inputs and are generally unknown. Neural networks use a class of algorithms based on a concept of inter-connected “neurons.” In a typical neural network, neurons have a given activation function that operates on the inputs. By determining proper connection weights (a process also referred to as “training”), a neural network achieves efficient recognition of a desired patterns, such as images and characters. Oftentimes, these neurons are grouped into “layers” in order to make connections between groups more obvious and to each computation of values. Training the neural network is a computationally intense process.